User association is the problem that a wireless terminal chooses to access one serving base station. User association can be seen as the first step in wireless resource management, which has an important impact on network performance, and plays a very important role in achieving load balance, interference control, improvement of spectrum and energy efficiency. Aiming at the characteristics of multi-layer heterogeneous network including macro base stations and full-duplex small base stations, a separate multiple access mechanism was considered, which means allowing a terminal access different and multiple base stations in the uplink and downlink, so as to realize the performance improvement. On this basis, the user association problem with separation of uplink and downlink multi-access in heterogeneous network was modeled into an evolutionary game problem, in which the users act as the players to perform the resource competition with each other, the access choices of base stations are strategies in the game, and every user wants to obtain the maximum of own effectiveness by the choice of strategy. Besides, a low-complex self-organized user association algorithm was designed based on evolutionary game and reinforcement learning. In the algorithm, the user was able to adjust the strategy according to the revenue of current strategy choice, and reached an equilibrium state in the end, realizing user fairness. Finally, extensive simulations were performed to verify the effectiveness of the proposed method.